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Abstract. Ethiopia has scaled up integrated community case management of childhood illness (iCCM) in most regions. 
We assessed the strength of iCCM implementation and the quality of care provided by health extension workers (HEWs). 
Data collectors observed HEWs' consultations with sick children and carried out gold standard re-examinations. Nearly all 
HEWs received training and supervision, and essential commodities were available. HEWs provided correct case manage- 
ment for 64% of children. The proportions of children correctly managed for pneumonia, diarrhea, and malnutrition were 
72%, 79%, and 59%, respectively. Only 34% of children with severe illness were correctly managed. Health posts saw an 
average of 16 sick children in the previous 1 month. These results show that iCCM can be implemented at scale and that 
community-based HEWs can correctly manage multiple illnesses. However, to increase the chances of impact on child 
mortality, management of severe illness and use of iCCM services must be improved. 



INTRODUCTION 

Pneumonia, diarrhea, and malaria are among the leading 
causes of mortality in children younger than 5 years old. 1 
Effective therapies for these conditions exist, 2 but children 
in poor rural communities often do not have access to formal 
healthcare, 3 ' 4 and coverage of these interventions remains 
low in many countries. 5 Delivery of care through community 
health workers (CHWs) can increase coverage of specific 
treatments 6 " 8 and lead to substantial reductions in child mor- 
tality. 9 " 13 The World Health Organization (WHO) and the 
United Nations Children's Fund (UNICEF) recommend 
integrated community case management (iCCM) of pneumo- 
nia, diarrhea, and malaria. 14 However, experience with large- 
scale implementation of iCCM remains limited. In 2010, just 
12 countries in sub-Saharan Africa were implementing CCM 
of at least three illnesses, and only 6 sub-Saharan African 
countries were implementing CCM of at least three illnesses 
in at least 50% of the country's districts. 15 Few rigorous 
assessments of iCCM implementation have been conducted, 
and the limited evidence on quality of care is mixed. 16 " 22 

As part of Ethiopia's Health Extension Program, over 
30,000 female health extension workers (HEWs) have been 
trained for 1 year and deployed to communities throughout 
the country to provide preventive and curative health ser- 
vices. There are typically two HEWs assigned to a kebele 
(subdistrict) with a population of approximately 5,000 people. 
HEWs usually provide clinical care in health posts but may 
also provide care in the homes of community members. 

After a national policy change supporting community- 
based treatment of childhood pneumonia by HEWs in early 
2010, Ethiopia has scaled up iCCM provided by HEWs in 
most regions of the country. Antibiotic therapy (with 
cotrimoxazole) for pneumonia and zinc for treatment of diar- 
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rhea have been added to the pre-existing CCM (routine 
CCM) program, which included management of childhood 
diarrhea (with oral rehydration salts [ORS] only), malaria, 
malnutrition, measles, ear infection, and anemia. Other than 
the addition of antibiotics and zinc, the iCCM program pro- 
vides a number of program supports, including a 6-day train- 
ing for HEWs on iCCM, strengthened supervision, improved 
supply chain management for essential commodities, and 
enhanced monitoring and evaluation. Definitions of key 
terms are shown in Box 1. 

The scale-up of iCCM was implemented in a phased man- 
ner in Oromia Region. This allowed for randomization of 
rural woredas (districts) in two zones into phase one inter- 
vention areas (iCCM) and phase two comparison areas (rou- 
tine CCM) for an independent evaluation of the impact of 
iCCM. In addition to the present study, data for the evalua- 
tion came from baseline and endline household surveys to 
measure child mortality and coverage of treatments for 
common childhood illnesses. 

This study was commissioned by UNICEF to assess 
whether iCCM implementation strength, quality of care, and 
use of services were sufficient to plausibly expect a sub- 
stantial impact from the intervention on child mortality at 
the population level. We conducted a cross-sectional sur- 
vey of health posts where HEWs work in intervention and 
comparison areas. The study had two main objectives: 
(1) measure and compare indicators of iCCM implemen- 
tation strength in intervention areas and routine CCM 
implementation strength in comparison areas and (2) 
assess the quality of iCCM services provided to sick 
children in intervention areas. Quality of care was not 
assessed in comparison areas because the primary pur- 
pose of the study was to assess implementation strength 
and quality of care of iCCM; also, funding for data col- 
lection in comparison areas was limited. 

This survey was the first to evaluate the scale-up of iCCM 
in Ethiopia and the first rigorous assessment of the quality of 
care provided by iCCM-trained HEWs. It was also one of 
the first studies to assess the quality of iCCM services in 
sub-Saharan Africa and the first study to assess integrated 
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Box 1 

Definitions of key terms 

Key terms 

Correct classification: All HEW classifications matched gold standard classifications. 

Correct treatment: All HEW treatments matched gold standard treatments, including correct dose, duration, and frequency. 
Coverage of therapeutic interventions: The proportion of children under 5 years old with an illness who received correct treatment of 
that illness. 

Eligible iCCM illness: Lethargy or unconsciousness, convulsions, not eating or drinking, fever/malaria, cough, fast/difficulty breathing, diarrhea, 

vomiting, ear problem, signs/history of measles, malnutrition, feeding problems, or anemia. 
General danger signs: Not able to drink/breastfeed, vomits everything, had convulsions, and lethargy. 

Implementation strength: This term refers to the quantity of effective program activities carried out to reduce child mortality. These activities 

include training, supportive supervision, and continued availability of essential iCCM commodities. 
iCCM : In general, iCCM refers to the concurrent management of more than one common childhood illness. iCCM in Ethiopia (implemented 

in study intervention areas) is integrated management by an HEW at the community level of all of the following childhood illnesses: 

pneumonia, diarrhea (ORS and zinc), malaria, malnutrition, measles, anemia, and ear infection. 
Major iCCM illnesses: Pneumonia, diarrhea, malaria, measles, malnutrition, and danger signs. 

Quality of care: We assessed quality based on whether HEWs correctly assessed, classified, treated, and referred children with iCCM illnesses 

and provided counseling to caretakers based on Ethiopia iCCM clinical guidelines. 
Routine CCM: The routine CCM program in Ethiopia (implemented in study comparison areas) includes CCM of diarrhea (ORS only), 

malaria, malnutrition, measles, ear infection, and anemia. Pneumonia cases are referred to health centers. 
Severe illness: A child was considered to have a severe illness if the child had any of the following signs: any general danger sign (not able to 

drink/breastfeed, vomits everything, convulsions, lethargic, or unconscious), severe pneumonia, diarrhea with severe dehydration, severe 

persistent diarrhea, persistent diarrhea, dysentery, very severe febrile disease, severe complicated measles, severe complicated malnutrition, 

or severe anemia. 

Supervision with clinical reinforcement: Supervision with observation of patient consultations or register review. 

Eleven key assessment tasks: Checked whether child is able to drink/breastfeed, checked whether child vomits everything, checked whether 
child has had convulsions, checked whether child has lethargy, checked for cough or fast/difficult breathing, checked for diarrhea, checked 
for fever, checked for edema, checked for low MUAC (> 6 months) or visible severe wasting (< 6 months), checked for palmar pallor, and 
checked child's vaccination status. 



management of childhood pneumonia, diarrhea, malaria, and 
malnutrition by CHWs. 

MATERIALS AND METHODS 

Study population. The survey was conducted in rural areas 
of Jimma and West Hararghe Zones of Oromia Region. 
Oromia is Ethiopia's largest region, with approximately 
30 million people. 23 Eighty-three percent of the population 
lives in rural areas, and the primary occupation is subsistence 
agriculture. 24 Malaria risk varies in the study areas, with 
about one-quarter of health posts in high-malaria risk areas, 
25% with no malaria, and the rest with low malaria risk. The 
study was conducted just before the start of the rainy, high- 
malaria transmission season. Jimma and West Hararghe 
Zones have populations of approximately 2.9 and 2.1 million 
people, respectively. 25 

Survey design and inclusion criteria. We conducted a cross- 
sectional survey of rural health posts in the two study zones. 
The sample was stratified by iCCM intervention and compar- 
ison areas. Intervention areas received the iCCM program 
detailed above, whereas comparison areas continued with the 
pre-existing routine CCM program. Table 1 details the differ- 
ences between the iCCM and the routine CCM programs. 
Systematic random sampling was used to select health posts 
within each stratum. 

All HEWs present and providing case management services 
in selected health posts were included. Children had to meet 
the following inclusion criteria: (1) 2-59 months of age (chil- 
dren younger than 2 months of age were excluded because 
there are separate clinical guidelines for this age group that 
call for referral to a health facility for most cases; also, we 
expected an extremely small sample of children in this age 



group, which would not justify the extra expense of training 
data collectors on the algorithm for children younger than 
2 months of age), (2) having at least one complaint consistent 
with an eligible iCCM illness, and (3) initial consultation for 
the current illness episode. 

Sample size. We selected 104 health posts from interven- 
tion areas and 46 health posts from comparison areas. Propor- 
tions of the indicators were assumed to be 50% to give the 
most conservative sample size. Confidence was set at 95%. 
We adjusted for non-response of 5% for health posts, 5% for 
HEWs, and 10% for patients. A design effect of 1.3 was 
included to account for clustering of HEWs and patients in 
health posts. Assuming an average of 1.5 HEWs per health 
post, 104 health posts allowed for estimates of health post- 
and HEW-level indicators in intervention areas with a preci- 
sion of ±10% points. Assuming two sick children per health 
post, primary patient-level indicators would have a precision 
of ±9% points. The smaller sample size in comparison areas 
allowed a precision of ±15% points for health post- and 
HEW-level indicators. 

Instruments and indicators. The survey instruments and 
primary indicators of implementation strength and quality of 
care were adapted from the WHO Health Facility Survey 
tool, 26 a survey of Health Surveillance Assistants in Malawi, 21 
and the CCM Global Indicators 27 Indicator definitions are 
presented in Supplemental Appendix 1. 

Data collection. Data were collected in May and June of 
2012, about 1 year after completion of training of HEWs in 
iCCM and the beginning of iCCM implementation in the inter- 
vention areas. Data collectors were health professionals who 
had worked as iCCM trainers or supervisors. Survey personnel 
were trained for 7 days, and all observers and re-examiners 
achieved at least 90% concordance with gold standard clinicians 
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Table 1 

Comparison of case management guidelines and program inputs for the Ethiopia iCCM program and routine CCM 



ICCM (intervention areas) 



Routine CCM (comparison areas) 



Management of iCCM illnesses for children 2-59 months 



Pneumonia 
Severe pneumonia 

Diarrhea (some dehydration, no dehydration) 
Severe diarrhea (severe dehydration, persistent 
diarrhea, severe persistent diarrhea, dysentery) 



Malaria 

Severe febrile disease 
Uncomplicated malnutrition 
Severe complicated malnutrition 

Measles 

Severe complicated measles 

Measles with eye or mouth complications 

Acute ear infection 

Anemia 
Program inputs 
Training 
Supervision 

Supply of commodities 



Monitoring and evaluation 



Cotrimoxazole 

Pre-referral treatment with cotrimoxazole; 

referral to health center 
ORS/ORT; zinc 

ORS; vitamin A (for persistent and severe persistent 
diarrhea only); referral to health center 



Antimalarial 

Pre-referral treatment with cotrimoxazole; 

referral to health center 
RUTF or supplementary feeding program 

Pre-referral treatment with amoxicillin 
and vitamin A; referral to health center 

Vitamin A 

Vitamin A; cotrimoxazole; tetracycline eye 
ointment (optional); referral to health center 

Vitamin A; tetracycline eye ointment (optional); 
gentian violet (optional) 

Paracetamol; referral to health center 

Referral to health center 

6 days of training on iCCM 

Standardized supportive supervision on iCCM 
supported by partner NGOs plus standard 
government supervision; biannual PRCM meetings 

Support for purchase and supply of drugs and other 
commodities by UNICEF and partners; provision 
of iCCM registers, iCCM chart booklets, timers, 
and other supplies 

Enhanced data collection during supervisions and PRCM 
meetings; data management support by UNICEF 



Referral to health center 
Referral to health center 

ORS/ORT 

ORS; vitamin A 

(for persistent and severe 
persistent diarrhea only); 
referral to health center 

Antimalarial 

Referral to health center 

RUTF or supplementary 

feeding program 
Pre-referral treatment with 

amoxicillin and vitamin A; 

referral to health center 
Vitamin A 

Vitamin A; tetracycline eye 

ointment (optional); referral 

to health center 
Vitamin A; tetracycline eye 

ointment (optional); 

gentian violet (optional) 
Paracetamol; referral to 

health center 
Referral to health center 

No additional training 

Standard government supervision 



Standard government commodity 
supply chain system; 
no additional supplies or 
job aids 

Standard government monitoring 
and evaluation 



NGO - non-governmental organization; ORT - oral rehydration salts; PRCM - performance review and clinical mentoring; RUTF - ready-to-use therapeutic food. 



on three consecutive role play examinations. Data collectors 
were sent to woredas in which they did not normally work to 
avoid biasing the results. 

Assessment of program implementation strength. In inter- 
vention and comparison areas, survey teams collected informa- 
tion on the strength of iCCM/routine CCM implementation. 
Data collectors inspected iCCM commodities, supplies, and 
job aids. Next, they conducted interviews with the HEWs to 
learn about training and supervision received as well as HEW 
characteristics. Finally, they recorded information on sick child 
consultations from patient registers. 

Assessment of quality of care. The assessment of quality of 
care was conducted in intervention areas only. HEWs were 
notified of upcoming study visits and asked to mobilize care- 
takers to bring sick children to the health post on the day of 
the survey team visit. If fewer than two children presented at 
the health post within 1 hour of the health post opening, the 
team supervisor, along with an HEW or community volun- 
teer, recruited sick children from nearby households. Eligible 
children received a consultation from an HEW while a data 
collector silently observed and recorded details of the HEWs 
assessment, classification, treatment, referral, and counseling. 
Next, the observer took the patient and caretaker away from 
the HEW and asked the caretaker to explain how they would 



administer any treatments prescribed for the child. Finally, 
the re-examiner performed a consultation with the patient 
and caretaker using a re-examination form that followed 
Ethiopia iCCM clinical guidelines. 

Data analysis. Data were entered directly into tablet com- 
puters using the Open Data Kit (ODK) 28 as the data capture 
software and stored in a Research Electronic Data Capture 
(REDCap) database. 29 All analyses were carried out in Stata 
12. 30 Descriptive statistics comprising proportions or means 
were calculated for selected indicators. Standard errors and 
associated 95% confidence intervals for HEW- and patient-level 
variables were calculated using the Taylor linearization method 
to account for clustering within health posts. 31 Indicators of 
correct classification or treatment were calculated by compar- 
ing the HEWs classification and treatment with the gold stan- 
dard classification from the re-examination and the associated 
treatments/referral recommended in the iCCM clinical guide- 
lines. We performed clinical pathways analyses stratified by 
children with severe and uncomplicated illness to identify where 
case management errors occurred during assessment, classifica- 
tion, treatment, and referral of sick children. 32 ' 33 The full 
research protocol and study instruments are available online. 34 

Ethical considerations. Ethical approval was obtained from 
the Institutional Review Boards (IRBs) of the Oromia Regional 
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Health Bureau and the Johns Hopkins University Bloomberg 
School of Public Health. Informed oral consent was obtained 
from all HEWs and caretakers of sick children. Written consent 
from all participants would not have been possible because 
many of the participants were not literate. The IRBs approved 
the use of oral consent, and data collectors noted on a form 
whether oral consent was obtained from each participant. 

RESULTS 

One health post in the intervention areas was excluded, 
because it was closed indefinitely, giving 103 health posts in 
intervention areas and 46 health posts in comparison areas 
that were successfully surveyed. All HEWs encountered in 
health posts were included in the study, resulting in samples 
of 137 HEWs in intervention areas and 64 HEWs in compar- 
ison areas. In total, 257 children were included in interven- 
tion areas. 

Table 2 shows the characteristics of the HEWs included in 
the sample. They were all women, and HEWs in both study 
arms had an average of about 4 years of experience as an 
HEW; 91% of HEWs in intervention areas and 86% of 
HEWs in comparison areas reported living in the same kebele 
in which they work. However, only 12% of HEWs in inter- 
vention areas and 6% of HEWs in comparison areas reported 
having lived in the kebele in which they work before begin- 
ning their initial HEW training, indicating that most HEWs 
were not selected from the communities in which they work. 

Table 3 presents the characteristics of sick children in the 
sample. According to the gold standard classifications, diar- 
rhea (66%), pneumonia (15%), malnutrition (13%), and ear 
infection (12%) were the most common illnesses. Few chil- 
dren presented with malaria (1%), measles (2%), or anemia 
(4%). Spontaneous consultations accounted for only 18% of 
the sample of children; 37% of children were mobilized by the 
HEWs, and active recruitment of sick children accounted for 
45% of the sample. 

Program implementation strength. Table 4 shows the pro- 
portion of HEWs that received training and the proportion 
of health posts that received supervision in intervention and 
comparison areas. Nearly all HEWs (98%, 95% confidence 



Table 2 

Distribution of HEWs by selected characteristics in intervention (N = 
137) and comparison (N = 64) areas in Jimma and West Hararghe 
Zones, Oromia Region, Ethiopia in 2012 

Intervention areas Comparison areas 

Characteristic n Percent n Percent 



Age, years 



18-20 


13 


9.5 


9 


14.1 


21-23 


83 


60.6 


35 


54.7 


24-26 


35 


25.6 


18 


28.1 


27-29 


4 


2.9 


1 


1.6 


30-32 


2 


1.5 


1 


1.6 


Marital status 










Married 


80 


58.4 


39 


60.9 


Single 


56 


40.9 


25 


39.1 


Separated/divorced 


1 


0.7 


0 


0.0 


HEW lives in the same 


125 


91.2 


55 


85.9 


kebele as health post 










HEW lived in the kebele 


16 


11.7 


4 


6.3 



1 year before completing 
basic HEW training 



Table 3 

Distribution of sick children by selected characteristics in 
intervention areas in Jimma and West Hararghe Zones, Oromia 
Region, Ethiopia in 2012 (N = 257) 



Characteristic 




Percent 


Age, months 






2-11 


94 


36.6 


12-23 


92 


35.8 


24-35 


39 


15.2 


36-47 


22 


8.6 


48-59 


10 


3.9 


Sex 






Male 


129 


50.2 


Female 


128 


49.8 


Gold standard disease classifications 






Pneumonia 


39 


15.2 


uiai l urjd 


169 


65.8 


Malaria/severe febrile disease 


3 


1.2 


Measles 


5 


2.0 


Malnutrition 


32 


12.5 


Ear infection 


30 


11.7 


Anemia 


11 


4.3 


Severe illness 


38 


14.8 


Needs referral* 


63 


24.5 


Method of recruitment 






Spontaneous 


45 


17.5 


Mobilized by HEWs 


96 


37.4 


Recruited by survey team 


116 


45.1 


*In addition to severe illnesses, acute ear infection and anemia require referral. 



interval [95% CI] = 93-99) in the intervention areas received 
the iCCM training. HEWs in 87% (95% CI = 79-93) of 
health posts received at least one supervision visit related to 
iCCM in the previous 3 months, and 85% (95% CI = 77-91) 
of health posts received supervision that included clinical rein- 
forcement (observation of consultations or register review 
with feedback). As expected, no HEWs in comparison 
areas had received the iCCM training. Only 43% (95% CI = 
28-59) of health posts in comparison areas had been super- 
vised on routine CCM in the previous 3 months, and 19% 
(95% CI = 9-34) of health posts received supervision with 
clinical reinforcement. 

Table 5 presents the proportions of health posts with key 
iCCM/routine CCM commodities, supplies, and job aids on 
the day of data collection and health posts with no stockout of 
more than 7 consecutive days in the previous 3 months in 
intervention and comparison areas; 69% (95% CI = 59-78) 
of intervention health posts had all seven essential commodi- 
ties for iCCM on the day of data collection. The proportion of 
health posts with individual items in stock ranged from 99% 
(cotrimoxazole) to 80% (ready-to-use therapeutic food). 
Only 4% (95% CI = 1-15) of health posts in comparison areas 
had all five essential commodities for routine CCM in stock. 
About one-half (52%, 95% CI = 42-61) of health posts in 
intervention areas and all health posts in comparison areas 
reported a stockout lasting longer than 7 consecutive days in 
the previous 3 months of at least one of the essential items. 
Just under one-half (46%, 95% CI = 36-56) of intervention 
health posts and no comparison health posts had all essential 
supplies and job aids in stock on the day of the visit. Nearly all 
indicators of implementation strength were significantly 
higher in intervention areas than comparison areas. 

Quality of care. Table 6 presents key indicators of quality of 
care in intervention areas. HEWs completed an average of 9.2 
of 11 key assessment tasks. A large majority of children (81%, 
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Table 4 

Selected indicators of training and supervision in intervention and comparison areas in Jimma and West Hararghe Zones, Oromia Region, 
Ethiopia in 2012 



Intervention areas 



Comparison areas 



Percent (95% CI) 



Percent (95% CI) 



P valuer 



HEW trained in iCCM 137 
Health post received 100§ 

supervision on iCCM/routine 

CCM in the previous 3 months 
Health post received supervision 100§ 

on iCCM/routine CCM that included 

register review or observation of 

consultations in the previous 3 months 
HEW received instruction in iCCM/routine 137 

CCM clinical practice at a health center 

in the previous 3 months 

* Number of HEWs or health posts eligible for indicator. 

tTwo-sample binomial test of difference in proportions between intervention and comparison areas. 

$HEWs in comparison areas were not expected to be trained in iCCM, and therefore, this result confirms that there was little to no spillover of iCCM training to HEWs outside of the 
intervention areas. 

§Three health posts were excluded, because HEWs reported not being present for the majority of the previous 3 months. 
1 Four health posts were excluded, because HEWs reported not being present for the majority of the previous 3 months. 



97.8 (93.3-99.3) 
87.0 (78.8-92.9) 



85.0 (76.5-91.4) 



57.7 (48.8-66.0) 



64* 
421 



421 



64 



0.0 

42.9 (27.7-59.0) 



19.1 (8.6-34.1) 



7.8 (3.1-18.3) 



< 0.001 

< 0.001 



< 0.001 



< 0.001 



95% CI = 74-86) was assessed for the presence of cough, 
diarrhea, fever, and malnutrition. Fewer children (62%, 95% 
CI = 53-70) were assessed for all four general danger signs. 
Just over one-half of children (53%, 95% CI = 46-60) were 
correctly classified for all major iCCM illnesses. 

Nearly two-thirds of children (64%, 95% CI = 57-71) were 
correctly managed for all major iCCM illnesses. HEWs cor- 
rectly treated 72% (95% CI = 56-84) of children with pneu- 
monia, 79% (95% CI = 72-85) of children with diarrhea, and 
59% (95% CI = 40-76) of children with malnutrition. Sample 
sizes of children with malaria and measles were too small to 
draw meaningful conclusions about management of those ill- 



nesses. Only 6% (95% CI = 3-10) of children received an 
antibiotic when it was not indicated, and no children received 
an unnecessary antimalarial. Just 34% (95% CI = 22-50) of 
children with severe illness were correctly managed, and 
HEWs referred about one-half (54%, 95% CI = 41-67) of 
children needing referral to a health center. Among 13 chil- 
dren with severe illness who were not referred, 8 (62%, 95% 
C= 31-85) children received correct treatment for their ill- 
nesses (excluding the requirement to refer). Furthermore, 
few children (14%, 95% CI = 8-21) received the first dose of 
all needed treatments in the presence of the HEW. Only 18% 
(95% CI = 10-31) of children needing a vitamin A supple- 



Table 5 

Availability of essential iCCM/routine CCM commodities, supplies, and job aids on the day of data collection and no stockout of essential 
commodities in the 3 months preceding the survey in intervention (N = 103) and comparison (N = 46) areas in Jimma and West Hararghe 
Zones, Oromia Region, Ethiopia in 2012 

Available on day of data collection No stockout > 7 days in the last 3 months 

Intervention areas Comparison areas Intervention areas Comparison areas 



Item 


n 


Percent (95% CI) 


n 


Percent (95% CI) 


P value* 




Percent (95% CI) 


n 


Percent (95% CI) 


P value 


All essential commodities 


71 


68.9 (59.1-77.7) 


2 


4.4 (0.5-14.8) 


< 0.001 


53 


51.5 (41.4-61.4) 


0 


0.0 


< 0.001 


for iCCM/routine CCMt 






















Cotrimoxazole* 


102 


99.0 (94.7-100) 


1 


2.2 


< 0.001 


102 


99.0 (94.7-100) 


2 


4.4 (0.5-14.8) 


< 0.001 


ORS 


100 


97.1 (91.7-99.4) 


28 


60.9 (45.4-74.9) 


< 0.001 


93 


90.3 (82.9-95.2) 


28 


60.9 (45.4-74.9) 


< 0.001 


Zinc* 


99 


96.1 (90.4-98.9) 


0 


0.0 


< 0.001 


83 


80.6 (71.6-87.7) 


0 


0.0 


< 0.001 


ACT 


91 


88.4 (80.5-93.8) 


23 


50.0 (34.9-65.1) 


< 0.001 


90 


87.4 (79.4-93.1) 


26 


56.5 (41.1-71.1) 


< 0.001 


Chloroquine 


92 


89.3 (81.7-94.5) 


17 


37.0 (23.2-52.5) 


< 0.001 


91 


88.4 (80.5-93.8) 


18 


39.1 (25.1-54.6) 


< 0.001 


RUTF 


82 


79.6 (70.5-86.9) 


16 


34.8 (21.4-50.2) 


< 0.001 


80 


77.7 (68.4-85.3) 


14 


30.4 (17.7-45.8) 


< 0.001 


RDT 


92 


89.3 (81.7-94.5) 


29 


63.0 (47.5-76.8) 


< 0.001 


91 


88.4 (80.5-93.8) 


29 


63.0 (47.5-76.8) 


< 0.001 


All essential supplies and job aids 


47 


45.6 (35.8-55.7) 


0 


0.0 


< 0.001 












for iCCM/routine CCM§ 






















Functional timer 


94 


91.3 (84.1-95.9) 


5 


10.9 (3.6-23.6) 


< 0.001 












Thermometer 


82 


79.6 (70.5-86.9) 


30 


65.2 (49.8-78.6) 


0.06 












Weighing scale 


79 


76.7 (67.3-84.5) 


37 


80.4 (66.1-90.6) 


0.612 












MUAC tape 


102 


99.0 (94.7-100) 


41 


89.1 (76.4-96.4) 


0.005 












Clean water 


74 


71.8 (63.0-80.7) 


3 


6.5 (1.4-17.9) 


< 0.001 












Supplies for ORS1 


77 


74.8 (65.2-82.8) 


0 


0.0 


< 0.001 












Chart booklet with clinical guidelines 


103 


100 


0 


0.0 


< 0.001 












Sick child register 


103 


100 


7 


15.2 (6.3-28.9) 


< 0.001 













ACT = artimisinin-based combination therapy; MUAC = middle upper arm circumference; RDT = rapid diagnostic test (malaria). 
* Two-sample binomial test of difference in proportions between intervention and comparison areas. 

f Intervention areas: cotrimoxazole, ORS, zinc, ACT, chloroquine, RUTF, and RDT. Comparison areas: ORS, ACT, chloroquine, RUTF, and RDT. 

t Cotrimoxazole and zinc were not part of the routine CCM program, and therefore, comparison health posts are not expected to have these drugs available. 

§Intervention areas: timer, thermometer, weighing scale, MUAC, clean water, supplies for ORS, iCCM chart booklet, and sick child register. Comparison areas: timer, thermometer, weighing 
scale, MUAC, clean water, supplies for ORS, and sick child register, 
f Cup and spoon. 
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Table 6 



Selected indicators of quality of case management by HEWs in intervention areas in Jimma and West Hararghe Zones, Oromia Region, Ethiopia 
in 2012 



Indicator 


N* 


Percent 


95% CI 


Assessment 








Child assessed for four general danger signs 


257 


61.9 


52.5-70.4 


Child checked for presence of cough, diarrhea, fever, and malnutrition 


257 


80.5 


73.6-86.0 


Child with cough or difficult breathing assessed for fast breathing by counting of respiratory rate 


148 


93.2 


85.8-96.9 


Child's vaccination status checked (children under 12 months) 


94 


97.9 


91.7-99.5 


Child's respiratory rate counted by the HEW within ± five breaths of the gold standard 


130 


70.0 


60.8-77.8 


Classification 








Child correctly classified for all major iCCM illnesses 


257 


52.9 


45.6-60.1 


Child not up to date on immunizations classified as not up to date 


77 


36.4 


26.1-48.1 


Treatment and referral 








Child correctly treatedt and/or referred for all major iCCM illnesses 


257 


64.2 


57.4-70.5 


Child with pneumonia correctly treated for pneumonia 


39 


71.8 


55.8-83.7 


Child with diarrhea correctly treated for diarrhea 


169 


79.3 


71.5-85.4 


Child with malnutrition correctly treated for malnutrition 


32 


59.4 


40.4-76.0 


Child with malaria correctly treated for malaria 


3 


66.7 


2.0-100 


Child with measles correctly treated for measles 


5 


20.0 


0.3-94.9 


Child with severe illness correctly treated and/or referred 


38 


34.2 


21.5-49.7 


Child needing referral correctly referred 


63 


54.0 


40.7-66.7 


CtiilH rpppivpd first Hnsp nf all nppdpH trpatmpnts in ttip nrp^PTipp rtf ttip l-TFW'i" 

V 1111^.1 1 Ll.t.1 V 111 l_l ! 1 1 V„ yjL till llk-LULU 1 1 ^ tl 1 1 1 H_ 1 1 1 .1 111 1111, L'l ClLllt-L V'l 111U 111 i VV -j- 


163 


13.5 


8.4-21.0 


Child needing vitamin A supplementation received vitamin A 


66 


18.2 


9.9-31.0 


Child needing mebendazole received mebendazole 


30 


20.0 


9.0-38.8 


Child received an unnecessary antibiotic 


257 


5.5 


3.0-9.7 


Child received an unnecessary antimalarial 


257 


0.0 




Counseling 








Caretaker received demonstration of how to administer all treatments by the HEWt 


160 


74.4 


63.4-82.9 


Caretaker correctly described how to give all treatments:]: 


156 


83.3 


75.5-89.0 


Caretaker advised to give extra fluids and continued feeding for diarrhea§ 


140 


85.0 


77.7-90.2 


Caretaker advised to return immediately if the child cannot drink/breastfeed or becomes sicker§ 


213 


36.2 


27.3-46.0 


Caretaker advised on when to return for follow-up§ 


213 


93.4 


88.0-96.5 



* Number of children eligible for task. 

t Includes prescription with correct dose, duration, and frequency. 

t Includes cotrimoxazole, ORS, zinc, vitamin A, ACT, chloroquine, and amoxicillin and excludes children who were referred. 
§ Excludes children who were referred. 



ment received vitamin A, and 20% (95% CI = 9-39) of chil- 
dren needing deworming medication received mebendazole. 
Over three-quarters of caretakers (74%, 95% CI = 63-83) 
received a demonstration on how to administer all treatments 
by the HEW, and 83% (95% CI = 76-89) of caretakers cor- 
rectly described how to give all treatments. 

Figures 1 and 2 present analyses of clinical errors for chil- 
dren with uncomplicated illness only and children with at 
least one severe illness in intervention areas, respectively. 
Around one-third of children were assessed for all 11 selected 
signs and symptoms of iCCM illnesses. The most common 
assessment errors were failures to assess convulsions, edema, 
and lethargy. Incorrect classification of pneumonia was the 
most common classification error for children with uncompli- 
cated illness, partially because of incorrect assessment of fast 
breathing. The most common treatment errors were failure to 
give cotrimoxazole to children with pneumonia and failure to 
give ORS for diarrhea, although these items were in stock. 
Misclassification was common among children with severe 
illness, and incorrect treatment was common regardless of 
whether children were correctly classified. The most common 
treatment errors for children with severe illness were failure 
to give the first dose of cotrimoxazole for pneumonia, failure 
to give the first doses of amoxicillin and vitamin A for severe 
complicated malnutrition, and not referring children to health 
centers when it was required. 

Use and service provision. Table 7 presents the mean num- 
ber of sick child consultations per health post in the previous 1 
month and selected indicators of service provision in interven- 
tion and comparison areas. Intervention health posts saw an 



average of 16 (95% CI = 13-19) sick children in the previous 1 
month (range = 0-95) compared with 5 (95% CI = 2-8) sick 
children seen in comparison health posts (range = 0-32). Vir- 
tually no children under 2 months of age were seen in inter- 
vention or comparison areas. 

When asked about their activities the day before the inter- 
view, HEWs in intervention areas reported spending an aver- 
age of 6.1 hours on work tasks. They spent an average of 
about 4 hours providing or offering clinical services in the 
health post, about 30 minutes providing clinical services in 
the community, and nearly 1 hour carrying out community 
mobilization/education activities. HEWs in comparison areas 
spent less time in health posts and more time in the commu- 
nity. Intervention and comparison health posts were report- 
edly open and offering clinical services for an average of 
about 20 hours in the previous week. 

DISCUSSION 

Several countries and international agencies are pushing for 
rapid scale-up of iCCM, despite a dearth of evidence on best 
practices for implementation, quality of care, and impact. The 
scale-up of iCCM in Ethiopia is the most ambitious imple- 
mentation of the strategy in sub-Saharan Africa to date. The 
country also has the advantage of a large literate, relatively 
well-trained, and salaried cadre of HEWs. Thus, it is impor- 
tant to take note of the lessons from the implementation of 
iCCM in Ethiopia for both improvement of the Ethiopia pro- 
gram and implementation in other countries as they prepare 
to scale up iCCM. 



430 



MILLER AND OTHERS 



Children with 


80 


uncomplicated 


(37%) 


illness only 


139 


(219) 


(63%) 



-Assessed 
convulsions: 
(73/139) 53% 
-Assessed edema: 
(32/139) 23% 





56 


Assessed for all 


(70%) 


11 key signs & 




symptoms 


24 
(30%) 



-Correct pneumonia 
classification: (14/24) 58% 
-Fast breathing correctly 

dassrfied: (16/24) 67K 



Not assessed 


69 


for all 11 key 


(50%) 


signs & 


70 


symptoms 


(50%) 



-Correct pneumonia 
classification: (35/70) 50% 
-Fast breathing correctly 

dassrfied: (43/70) 61% 





53 


Correctly 


(95%) 


classified 


3 




(5%) 





15 




(63%) 


Incorrectly 


7 


classified 


(29%) 




2 




(8%) 





53 




(77%) 


Correctly 


14 


classified 


(20%) 




2 




(3%) 





31 




(44%) 


Incorrectly 


35 


classified 


(50%) 




4 




(6%) 



Correctly 
treated: 
(152/219) 69% 



Incorrectly 
treated:* 
(59/219) 27% 



Not treated: 
(8/219) 4% 



-Child with pneumonia 
given cotrimoxazole: 

(7/14) 50% 
-Child with diarrhea 
given ORS or ORT: 
(24/43) 56% 



' Child was given at least one treatment, but not all HEW treatments matched gold standard treatments. 

Figure 1. Clinical errors analysis for children with uncomplicated illnesses in intervention areas in Jimma and West Hararghe Zones, Oromia 
Region, Ethiopia. ORT = oral rehydration therapy. 



iCCM has largely been implemented as planned in the 
study areas. Nearly all HEWs were trained in iCCM, and 
87% had received supervision in iCCM in the previous 
3 months. Nearly 70% of health posts had all essential com- 
modities for iCCM. 

HEWs in iCCM intervention areas performed most basic 
assessment tasks and correctly managed nearly two-thirds of 
all children, with minimal overprescription of drugs. Compar- 
isons of our results with other studies with similar assessment 
methods show that adherence to clinical guidelines seems to 
be at least as high for HEWs as for health workers at hospi- 
tals and health centers in Ethiopia. 35 Quality of care pro- 
vided by HEWs also compares favorably with performance 
in management of multiple childhood illnesses by CHWs 
in Malawi, 21 Kenya, and Papua New Guinea. 16 A recent 
review of studies of CCM of pneumonia in sub-Saharan 
Africa concluded that CHWs had difficulties with counting 
children's respiratory rate, correct classification and treat- 
ment of pneumonia, and overuse of antibiotics. 22 We found 
that HEWs counted respiratory rates more accurately, cor- 
rectly treated a higher proportion of children with pneumo- 
nia, and overprescribed antibiotics less than CHWs in the 
previous studies. 

Despite these achievements, only about one-third of chil- 
dren with severe illness were correctly managed, which is 
consistent with prior research. 17 ' 36 An important reason for 
this result was that only about one-half of children needing 



referral to a health facility were referred by the HEW. Other 
studies have shown that many referred children never reached 
the referral facility or caretakers complied with referral only 
after substantial delays. 37,3 Lack of transportation, cost of 
transportation, fees for care at referral facilities, and respon- 
sibilities at home can all discourage compliance with refer- 
ral. 39 ' 0 The failure of HEWs to refer children with severe 
illness seen in this study may have been attributable to the 
fact that the HEWs did not believe the caretakers would 
comply, and therefore, they preferred to treat the children in 
the health post. 

Studies from Pakistan have shown that CHWs can diagnose 
and treat severe pneumonia and that CCM of severe pneumo- 
nia can reduce treatment failures. 41 ' 42 However, more evi- 
dence is needed, especially from sub-Saharan Africa. 
Research is also needed on the reasons for mismanagement 
of children with severe illness and whether health outcomes 
can most feasibly and effectively be improved through 
enhanced training of CHWs on recognition of severe illness 
and danger signs, facilitation of transportation for referral, or 
enabling CHWs to treat children with severe illness in the 
community. Although mismanagement of children with 
severe illness is a serious concern, it should be noted that, 
even if referral rates are low, iCCM can still contribute to 
substantial reductions in mortality through the early manage- 
ment of illnesses, thus preventing children from becoming 
severely ill in the first place. 
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Children with 
severe illness 
(38) 



26 
(68%) 



-Assessed convulsions: 
(10/26) 38% 
-Assessed lethargy: 

(13/26) 50% 
-Assessed edema: 
(2/26) 8% 





6 


Assessed for all 


(50%) 


11 key signs & 




symptoms 


6 

(50%) 



-Correct diarrhea 
classification: 
(2/6) 33% 



Not assessed 


5 


for all 11 key 


(19%) 


signs & 


21 


symptoms 


(81%) 



-Correct diarrhea 
classification: 
(6/21)29% 





3 


Correctly 


(50%) 


classified 


3 




(50%) 





4 


Incorrectly 


(67%) 


classified 


2 




(33%) 



Correctly 
treated and 

referred: 
(13/38) 34% 



Incorrectly 
treated* and 

referred: 
(12/38) 32% 



Correctly 


4 

(80%) 




classified 


1 




Incorrectly 




(20%) 


— T — T * 


treated and not 






referred: 
(12/38) 32% 




6 

(29%) 










5 

(24%) 




Incorrectly 






classified 


9 

(43%) 




Not treated: 
(1/38) 3% 




1 

(5%) 











onia 

given first dose of 
cotrimoxazole: (3/12) 25% 
-Child with malnutrition 
given first dose of 
amoxicillin: (1/12)8% 
-Child with malnutrition 
given first dose of vitamin 
A: (6/12) 50% 



-Child with pneumonia 

given first dose of 
cotrimoxazole: (0/11) 0% 
-Child with malnutrition 
given first dose of 
amoxicillin: (0/3)0% 
-Child with malnutrition 
given first dose of vitamin 
A: (0/3) 0% 



B Child was given at least one treatment, but not all HEW treatments matched gold standard treatments. 

Figure 2. Clinical errors analysis for children with at least one severe illness in intervention areas in Jimma and West Hararghe Zones, Oromia 
Region, Ethiopia. 



Few children accessed care from HEWs, and virtually no 
children under 2 months of age, the age group with the highest 
risk of mortality, were seen. Box 2 presents an estimation of 
the gap between expected and actual sick child consultations 
in intervention areas. Using the average number of consulta- 
tions in the previous 1 month and the estimated number of 
children under 5 years old in the health post catchment area at 
surveyed health posts, there were 0.26 consultations at health 
posts per child per year. If we assume that patients should 
seek care from HEWs for 25-50% of illness episodes (a con- 
servative estimate considering that HEWs are meant to be the 
primary source of care in the public health system and that 
care-seeking from other sources is low 43 ), only 16-38% of 
expected consultations from HEWs were actually seen. More 
encouragingly, significantly higher use in intervention areas 
than comparison areas suggests that use may have increased 
where the iCCM program was introduced and services 
improved. Furthermore, there was a large range in the num- 
ber of consultations in the previous 1 month, with several 
health posts registering two to three times the average. More 
research is needed to understand why some health posts have 
much higher use than others. 

Low use of child health services is a common challenge. It 
was seen in the multicountry evaluation of integrated man- 
agement of childhood illness (IMCI), where improvements in 
quality of care were mitigated by low levels of care-seeking at 
health facilities and a failure to reach sick young infants 
(younger than 2 months old). 44 " 46 Although CCM is meant to 
substantially increase access to care, uptake of community- 



based services is often disappointing. A global review of data 
on sources of care for sick children from Demographic and 
Health Surveys (DHSs) and Multiple Indicator Cluster Sur- 
veys (MICS) in 42 countries in sub-Saharan Africa and Asia 
found that CHWs generally were not an important source 

c 47 

of care. 

To earn the trust of the communities and attain greater 
uptake of services, program implementers must ensure con- 
sistent availability of health workers and essential commodi- 
ties. Demand creation activities and increased active case 
detection by HEWs are also urgently needed. The experience 
of recruiting children for this survey suggests that there are 
large numbers of children with easy access to HEWs who fail 
to use their services despite being seriously ill. The Ethiopian 
Government's launch of the "Health Development Army," 
for which up to 3 million community members will be 
mobilized nationally to serve as community health volunteers, 
presents a promising platform through which to carry out 
large-scale demand creation 48 However, a thorough evalua- 
tion of this new initiative is needed. 

This study has a number of limitations. First, small sample 
sizes of children with malaria and measles precluded assess- 
ment of management of those illnesses. Second, HEWs may 
have performed better under observation than they normally 
would have performed. 49 50 Third, informing HEWs of survey 
visits may have allowed them to prepare for the visits, which 
would have further biased the results positively. Fourth, 
recruitment and mobilization of sick children may have pro- 
vided a sample of patients that was different from spontaneous 
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Table 7 



Mean number of sick child consultations per health post and selected indicators of service provision in 
Jimma and West Hararghe Zones, Oromia Region, Ethiopia in 20f2 


intervention and comparison areas in 


Indicator 


Intervention areas 




Comparison areas 




Mean (95% CI) 


ange 


Mean (95% CI) 


ange 


Sick child consultations in previous 1 month 










Total 


16.0 (13.2-18.8) 


0-95 


5.0 (2.3-7.7) 


0-32 


0 to < 2 months 


0.3 (0.1-0.5) 


0-9 


0.03 (0.0-0.1) 


0-1 


2-59 months 


15.7 (13.0-18.4) 


0-94 


5.0 (2.4-7.6) 


0-31 


Female 


8.0 (6.6-9.5) 


0-40 


2.3 (0.7-3.9) 


0-19 


Male 


7.9 (6.4-9.4) 


0-57 


? 4 ( 1 2-3 1\ 


0-13 


Unspecified 


0.1 (0.0-0.1) 


0-4 


0.3 (0.0-0.7) 


0-6 


Hours health post was open in previous 1 week 


23.3 (21.0-25.5) 


0-40 


20.2 (17.0-23.5) 


0-40 


Hours spent by the HEW in the previous 1 day 










Providing clinical services in the health post 


4.0 (3.5-4.5) 


0-10 


1.8 (1.1-2.5) 


0-8 


Providing clinical services in the community 


0.5 (0.3-0.6) 


0-7.5 


0.8 (0.3-1.2) 


0-5 


Community education/mobilization; disease prevention 


0.9 (0.6-1.1) 


0-8 


1.5 (0.9-2.2) 


0-8 


Other health-related activities 


0.8 (0.5-1.1) 


0-8 


1.4 (0.7-2.1) 


0-8 


Other non-health-related activities 


0.2 (0.1-0.4) 


0-8 


0.4 (0.1-0.8) 


0-7 


Travel outside kebele 


0.7 (0.3-1.2) 


0-12 


0.5 (0.1-0.9) 


0-8 


Total work-related activities 


6.1 (5.6-6.6) 


0-11 


5.5 (4.9-6.2) 


0-9 



patients, although concerns that we would obtain few chil- 
dren with severe illness proved unfounded. If these children 
were different from the children that the HEWs normally 
would see, it could have biased the results. Fifth, some infor- 
mation was based on self -report by HEWs, which may be 
biased. Sixth, iCCM program implementers knew which 
woredas were selected as intervention areas; therefore, extra 
effort may have been given to these areas, and implementa- 
tion may have been stronger than in other areas of the coun- 
try. Seventh, because the HEWs are a relatively well-trained 
and well-remunerated cadre of health workers, the results on 
HEW performance may not be generalizable to CHWs with a 
lower skill level and fewer incentives. These limitations may 
have resulted in observed levels of implementation strength 
and quality of care that were higher than they would have 
been under normal circumstances. 

Scaling up iCCM at a national or regional level poses con- 
siderable challenges. The scale of the task of deploying CHWs 
and ensuring adequate training, sustained supervision, and 
availability of essential commodities can be daunting, but this 
challenge is well-recognized. The additional challenge of ensur- 
ing community demand for iCCM services is equally important 



but may be overlooked. Without concerted efforts to promote 
use of services, millions of dollars spent on implementation of 
iCCM may have little impact on child mortality. 

Received December 18, 2013. Accepted for publication March 28, 2014. 
Published online May 5, 2014. 

Note: Supplemental appendix appears at www.ajtmh.org. 

Acknowledgments: The authors thank Ato Shallo Dhaba and 
Dr. Zelalem Habtamu of the Oromia Regional Health Bureau and 
Dr. Luwei Pearson of the United Nations Children's Fund (UNICEF) 
Ethiopia Country Office for their strong support of this research. 
Thanks to ABH Services, PLC for implementation of the survey. We 
also thank the Ethiopian Federal Ministry of Health, the JSI Research 
and Training Institute, Inc./Last 10 Kilometers Project (ISI/L10K), 
the Integrated Family Health Program (JSI/IFHP), and UNICEF 
New York for their support and assistance. 

Financial support: This work was supported by United Nations 
Children's Fund (UNICEF) Award 1000017212 and Government 
of Canada Award 7056791. N.P.M. received funding from US 
National Institutes of Health Training Grant for Maternal and 
Child Health HD046405. 

Disclaimer: H.L. and T.D. were staff of the United Nations Children's 
Fund (UNICEF) Ethiopia Country Office. UNICEF was one of the 



Box 2 

Estimation of the gap between expected and actual iCCM consultations in intervention health posts in Jimma and West Hararghe Zones, 
Oromia Region, Ethiopia in 2012 

To illustrate the gap between expected and actual rates of use of iCCM services in intervention areas we conducted an estimation exercise using the following assumptions 

There were on average 744 children under 5 years old per health post in intervention health posts in Jimma and West Hararghe according to 
statistics available in the health posts. 

On average an under 5-year-old child is expected to have 0.28 episodes of pneumonia (based on estimated incidence among children 0-4 years 
old in Ethiopia in 2010) 51 3.3 episodes of diarrhea (based on estimated incidence among children under 5 years old in the WHO AFRO 
region in 2010) 52 and 0.17 episodes of malaria per year (based on incidence among children under 10 years old in Jimma Zone Oromia 
Region Ethiopia from 2008 to 2010). 53 Thus, the average total number of episodes of these three illnesses in 1 year for one child would 
be 3.75. For a 1-year period, this would equal a total of 2,790 episodes per health post catchment area (excluding children with malnutrition 
and other illnesses included in the iCCM program). 

If we assume that caretakers should seek care for 25-50% of those episodes from health posts/HEWs (others may seek care from other 
appropriate sources*) then we would expect 698-1,395 consultations for three illnesses per health post per year. 

The current number of iCCM consultations per intervention health post per year based on use in 1 month before data collection is 192. 

With 192 consultations per year for 744 children there are only 0.26 consultations per child per year at health posts. 

Using the assumptions above there is a gap of 506-1,203 consultations per health post per year. In other words only 14-28% of expected 
consultations with HEWs are being seen. 

*The iCCM evaluation baseline survey in Jimma and West Hararghe in 2011 found that 12.2%, 7.1%, and 11.6% of children with pneumonia, diarrhea, and fever, respectively, sought care at a 
public health facility other than HEW/health post. 
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